Safety culture in French nursing homes: A randomised controlled study to evaluate the effectiveness of a risk management intervention associated with care

Background French Nursing Homes (NHs) are in the early stages of implementing their Risk Management (RM) approach. A regional structure, which was mandated to provide independent support in RM, designed a training package. Objective To study the impact of the RM training package on safety culture (SC) in NHs and drivers for improvement in SC scores. Method and analysis This randomised controlled study targeted French NHs. Inclusion criteria were voluntary participation, no external support provided on the topic of adverse incidents upstream of the project, and the commitment of top management to its implementation. The 61 NHs were randomly allocated to one of two groups: the first benefited from a training package; support was given to the second after the impact measurement. Seven dimensions of SC were measured, at an 18-month interval, using the validated Nursing Home Survey on Patient Safety Culture questionnaire (22 items), which was administered to all of the professionals working in NHs. Eleven variables were captured, relating to the structural profile of the NH, the choices of top management in terms of healthcare safety, and the implementation of the system. Further modelling identified predictive factors for changes in SC scores. Results 95% of NHs completed both rounds of the questionnaire. The dimension Feedback and communication about incidents (SC = 85.4% before the intervention) significantly improved (+2.8%; p = 0.044). Improvement in the dimension Overall perceptions of resident safety–organizational learning was close to significant (+3.1%; p = 0.075). Drivers for improvement in scores were a pre-existing quality improvement approach, and a steering group that showed RM leadership. Conclusions The system appears to have improved several dimensions of SC. Our findings are all the more important given the current crisis in the healthcare sector. Trial registration Retrospectively registered as NCT02908373 (September 21, 2016).

training, the creation of patient safety teams, or patient safety education programs implemented in hospitals-and even fewer have been evaluated for their effectiveness [15]. Moreover, few target NHs [16] or care at home [17].
As, to the best of our knowledge, there are no reports of schemes that propose a comprehensive RM structure tailored to NHs, QualiREL Santé (a RSS dedicated to the quality and safety of care) designed a training package to support French NHs in the implementation of their RM system. The scheme aimed to provide a long-term strategy for structuring and reporting AEAC of different levels of severity, and was based on the four axes found in the Shortell model [8]. It targeted staff at all hierarchical levels, whatever their role in supporting the elderly person. The aim was to develop SC among professionals and, more specifically, improve overall perceptions of resident safety. It was therefore important to study the impact of the implementation of the intervention on SC.
Any recommendations for improvements to the support system, or its successful implementation, must be underpinned by an understanding of the parameters that influence the final results [18][19][20]. These parameters may be related to the structural profile of the NH, its appropriation of the system, or the strategic healthcare safety choices made by top management [21].
The main objective of our research was, therefore, to study the impact of the implementation of an RM support system, which was deployed as training package, on the SC of professionals. We measured the evolution of SC in a group of NHs supported in their RM strategy; and the relation between the change in SC scores and the characteristics of the NH. Our results provide an insight into the actions and strategies that could be adopted to develop SC in NHs.

Participating nursing homes
A total of 61 NHs from two French administrative divisions (départements) (Loire-Atlantique and Vendée) participated. Inclusion criteria were: i) voluntary participation in September 2015 to benefit from a system to support the implementation of a RM approach, ii) a written commitment by the NH's top management to implement a RM approach; and iii) no external support had been provided by QualiREL Santé on the topic of adverse incidents upstream of the project. The characteristics of French NHs have been described in earlier work [21], and their legal status can vary (independent, part of a group, attached to a public hospital). They also differ in terms of the number of beds, the number of staff, and the ratio of staff to residents. High-dependency residents are scored according to their degree of dependency [22].

Tools
The training package. The overall objective of the training package was to outline a strategy to structure the reporting and analysis of AEAC in the long term. In particular, the goals were to: i) provide a definition of AEAC tailored to the context of French NHs, with illustrative examples [6]; ii) to propose scales for the severity and frequency of AEAC [23]; iii) to provide formal tools for a reporting and analysis system; and, iv) to create a steering group of NH professionals that would be responsible for the analysis of AEACs and the definition of an action plan.
The training package was expected to lead to improvements in SC among NH staff, particularly overall perceptions of resident safety. It consisted of 20 hours (for each NH) of methodological support provided by two facilitators from the research team. Support was structured into four interventions. The initial session [1] raised awareness among top management of the importance of taking ownership of the project. The aim of the second session [2] was to raise awareness among all NH staff of the need to report AEAC, and identify volunteers who would join the steering group. The third session [3] was run for members of the RM steering group; here the aim was to train them in how to structure their work and analyse AEAC. Then, the aim of the final session (4) was to observe how the steering group handled a critical AEAC that had occurred in the NH, and provide support in its analysis. At the end of each intervention, dedicated tools that had been specifically designed for the package were presented, and made available to participants. Table 1 presents the training package as a function of the four axes in Shortell's model [8]. The four sessions were implemented over a period of nine months.
Our recommendations for the implementation of the training package were as follows: as many NH staff (including top management) as possible should attend the awareness-raising session; the RM steering group should be composed of different staff categories (nurses, local doctors, care assistants, staff responsible for catering and laundry, secretaries, etc.); and the steering group should have met at least once (and, if possible, should have run an analysis of an AEAC that had occurred in the NH) before taking part in the fourth session.
The Nursing Home Survey on Patient Safety Culture. The French version of the psychometrically-validated NHSOPS includes 22 items that measure the following seven dimensions of SC [13]. Dimension 1 (Overall perceptions of resident safety and organizational learning) explores the ability of the facility to take account of its mistakes and make changes, and the overall quality of care provided for residents. Dimension 2 (Handoffs) assesses communication for resident care both within the NH, and with external structures. Dimension 3 (Teamwork) explores support, mutual respect and collaboration among colleagues. Dimension 4 (Supervisor expectations and actions promoting resident safety) captures supervisors' acknowledgement of compliance with procedures, and their openness to ideas for improvement. Dimension 5 (Compliance with procedures) explores general compliance with mandatory procedures implemented in the facility, regardless of any operational difficulties encountered by professionals. Dimension 6 (Staffing) captures the number of staff in relation to the workload. Finally, This self-administered questionnaire is aimed at professionals working in French NHs (both staff employed directly by the NH, and external contracted professionals). It is divided into four sections: "Working in your NH", "Communication", "Your hierarchy" and "Your NH". There are 22 items, which are scored on a 5-point Likert scale (never, rarely, sometimes, most of the time, always-or-do not agree at all, do not agree, agree up to a point, agree, completely agree). "Does not apply" and "Don't know" are other response categories. Finally, the questionnaire includes items relating to the sociodemographic characteristics of respondents.

Study design
The study followed the randomised controlled study design. It ran from September 2015 to November 2017.
Randomization. The 61 NHs were randomly allocated into two groups; the first group received support in 2016 (experimental group-31 NHs), and the second group received support after the impact assessment in November 2017 (control group-30 NHs). The homogeneity of the two groups in terms of distribution by département and legal status was checked.
Implementation of the training package. The operational implementation of the support system ran over a period of nine months. Two facilitators from the research team spent time in each NH. The latter had a training and coaching role in the implementation of the support system (from sessions 1 to 4). Two professionals were appointed by the NH to be the main point of contact during the project. SC measurement. The same NHSOPS-F questionnaire was administered to the two groups prior to the implementation of the training package (January 2016), and again around 18 months later (September 2017). It was important that the NH took ownership of the support system over time. The methodology for the distribution and collection of questionnaires has been extensively described in an earlier article [21]. To summarize, in each NH the two contact points were responsible for: distributing the questionnaire to each professional working in the NH; promoting and following-up the survey; centralizing the completed questionnaires; and returning them to the research team [21]. The survey was conducted over a sixweek period. Each completed questionnaire was given a number to ensure anonymity. Responses were recorded by an external technician, and data capture was blind, in duplicate, with cross-checking. Because SC was measured at the NH level, the analysis was performed by compiling all of the responses from professionals working in the same NH, with the aim of comparing responses from the same NH between the two measures.

Descriptive variables for NHs in the experimental group
Eleven descriptive variables were collected at the beginning of the project in 2016. Five related to the structure of the NH: bed capacity; legal status; being part of a group or not; the mean dependency score; and the staff/ resident ratio. Six other variables documented top managers' strategic choices in terms of safety of care: an established policy of ongoing improvement in quality and RM; an active quality improvement approach; and an active RM approach, approaches facilitated by the presence of either: an appointed RM quality officer (a member of staff with no specialized RM quality qualifications), a RM quality specialist (a graduate with RM quality qualifications), or an external quality and RM service provider. The categorization of these variables is presented in Table 2.

Change in descriptive variables for NHs in the experimental group in 2017
In 2017, change in scores for two variables related to the structural profile of the NH were captured: the staff/ resident ratio, and the mean dependency score ( Table 2).

Change in variables related to the implementation of the training package in the experimental group in 2017
As the project progressed, the following seven variables related to recommendations for implementing the system were measured.
1. The percentage of staff who attended the awareness-raising session.
2. The presence of top management at the awareness-raising session.
3. An anonymous questionnaire was distributed to each member of the steering group at the end of session 3. This questionnaire identified the percentage of professionals who "wanted to use the knowledge they had gained" or 4. who "felt able to pass on their knowledge to colleagues".
5. The degree of maturity of the steering group in implementing the approach was recorded by the two facilitators. This measure corresponded to the use of the systemic AEAC analysis tool that was made available between the 3rd and 4th sessions.
6. Facilitators also recorded the stability of the steering group, notably the withdrawal or replacement of its members.
7. Finally, the research team noted any contextual elements that could undermine the implementation of the system (for example, the relocation of the NH, turnover among top management, absenteeism among local management, and a lack of commitment from top management).
The categorization of these seven variables is shown in Table 2. The research team developed bespoke tools to collect this data, and these tools are not licensed.

Data analysis
The calculation of SC scores has been described in detail in previous work [21]. All analyses were performed at the NH level. Following the method given in the original American version of the NHSOPS [9], scores were calculated as the average percentage of positive responses on all items in each dimension.
The validity of the model constructed from data collected from the first round of questionnaires [13] was rechecked against data from the second round. Four statistical indices were taken into account in order to verify the fit of the model to the data: Steiger's Root Mean Square Error of Approximation (RMSEA), and the Standardized Root Mean Squared Residual (SRMR): fit was considered good when these values were <0.1, and very good when <0.05. The Tucker Lewis Index (TLI) and the Comparative Fit Index (CFI) were considered an acceptable fit when >0.9 [24]. Cronbach's α coefficients were calculated to assess internal consistency [25]. Consistency was considered acceptable when α>0.70 [26]. SC scores were calculated for each NH that completed the two questionnaires in 2016 (baseline) and in 2017 (following the intervention). Scores for the seven dimensions for each NH were compiled at the group level (experimental group = 30 NHs; control group = 28 NHs). Thresholds to discriminate the levels of development of the dimensions were set at: SC <50% (underdeveloped); 50% � SC <75% (developing); and SC � 75% (developed) (13).
Changes in scores for the seven dimensions were evaluated using a paired Student's t-test. A threshold of 0.05 was used to establish a significant difference between the two measures.
Descriptive statistics were run for all variables. Linear regression models were used to assess variables associated with each change in the SC score in univariate analyses to identify potential predictors at a significance level of 0.2. Multivariate analyses were then performed according to the backward stepwise method, in order to identify models that best fitted the data, and keep the most relevant factors associated with each change in the SC score, at a significance level of 0.05. Validity and goodness of fit were checked using adjusted R 2 and an F-test.
All statistical analyses were performed using R software (version 4.1.1).

Ethics approval and consent to participate
The

NH participation
Participation in the implementation of the training package. By November 2017, 30 of the 31 NHs in the intervention group had benefited from the entire support package (Fig 1).
Participation in the NHSOPS-F questionnaire. Out of the 61 NHs operating in 2015, 58 (95%) completed the two rounds of questionnaires (Fig 1). The questionnaire was administered to 3,390 professionals in 2016 (1,946 responded) and 3,140 professionals in 2017 (1,621 responded). Of those who responded in 2016, 1,012 were in the experimental group and 934 were in the control group. In 2017, these figures were 754 (experimental group) and 867 (control group). The median overall participation rate for all NHs was 64% in 2016 and 56.5% in 2017. S1 Table compares the characteristics of respondents in 2016 and 2017. The main differences between 2016 and 2017 concern: the percentage of paramedical professionals (+3.6%; p = 0.038) and doctors (−1.1%; p = 0.001); the percentage of professionals aged under 25 (+2.3%; p = 0.002) and between 36 and 45 (−3.1%; p = 0.026); and professionals working 15 hours or less per week (−2%; p = 0.007). All other categories were not statistically different at the 5% alpha level.

Descriptive variables
Descriptive variables for NHs in the experimental group. Of the 28 NHs in the experimental group that completed the two questionnaires, seven were hospital-based, and 21 had another status. Bed capacity was equal to or over 80 for 15 NHs. The mean staff/ resident ratio was 0.74 in 2016 and 0.77 in 2017. The mean dependency score was 625 in 2016 and 639.3 in 2017. In 2016, an active Quality improvement approach was in place in 24 NHs, and there was an active RM approach in 17. For 12 NHs, these policies were facilitated by a specialized RM Quality graduate, while in 21 NHs they were the responsibility of a salaried NH professional with no specific RM Quality qualifications. These results are presented in Table 2.
Change in descriptive variables for NHs in the experimental group. The analysis found that in the 28 NHs in the experimental group, there was no significant change in the staff/ resident ratio, or the mean dependency score during the research period (Table 2).
Variables related to the implementation of the training package. On average, 49.5% of NH professionals attended the awareness-raising session; top management were present in 60.7% of NHs. On average, 7.2 professionals were involved in the steering group, and on average, 96.2% of steering group members completed the questionnaire that was administered after session 3. The analysis found that 91.1% of steering group members wanted to put into practice the knowledge they had acquired during the training sessions, and 34% showed leadership in the RM process. Finally, following session 4, the steering group was considered mature in 57.2% of NHs, and stable in 75%. Table 2 documents the seven variables used to measure the impact of the intervention.

SC scores, group homogeneity
The validity of the model constructed in 2016 was confirmed by data from 2017 (S2 Table). Table 3 presents scores for the seven SC dimensions for experimental and control groups in 2016 and 2017. The analysis found no statistically significant differences between the two groups at the beginning of the project. Fig 2 reports changes in SC scores between the administration of the two questionnaires in experimental and control groups. There were no significant changes in the control group between the two runs. However, in the experimental group, scores for Dimension 7 (Feedback and communication about incidents; p = 0.044) did increase significantly, indicating an improvement. Scores for two other dimensions (Dimensions 1 and 4; Overall perceptions of resident safety-organizational learning and Supervisor expectations and actions promoting resident safety) also increased, and were close to significance (p = 0.075 and 0.067) (Fig 2). Finally, scores for Dimension 2 (Handoffs) fell, and the fall was close to significant (p = 0.089).

Relationships between variables and change in SC scores
The results of the final multivariate models are presented in Table 4, and details of the univariate models used to construct them are presented in S3 Table. The analysis found that scores for Dimension 1 (Overall perceptions of resident safety-organizational learning) improved significantly as a function of the percentage of members of the steering group who showed leadership, and in NHs with an active Quality improvement approach (respectively, βcoefficient = 0.14/

Discussion
The main objective of this article was to study the impact of the implementation of an RM support system on SC among NH professionals. The results show that there is a difference in SC scores between 2016 and 2017 for NHs in the experimental group, while there is no difference for NHs in the control group. This suggests that the intervention had an impact on the SC of the professionals who participated.

Change in SC scores
The intervention was designed to improve scores for Dimension 1 (Overall perceptions of resident safety) included in the original American version of the questionnaire [9]. The validation of the French version that was carried out during the research project led to the aggregation of this dimension with the dimension Organizational learning. The new Dimension 1 (Overall perceptions of resident safety-organizational learning), developed at the beginning of the project (75.8%), improved (by +3.1%) after the implementation of the support system. For the research team, the system clearly improved scores for the theme Overall perceptions of resident safety, however, the additional theme prevented the identification of a clear improvement in the new, aggregated dimension. Moreover, the literature underlines that organizational learning requires a process of profound change at different levels of the NH [15], and that its development can only be measured on the scale of many years [2]. Improvements in one other dimension deserve to be highlighted. Dimension 7 (Feedback and communication about incidents), which was already developed at the beginning of the project (85.4%), significantly improved (by +2.8%). This change was most probably brought about by, among other actions, the awareness-raising sessions, the tools that were made available, and empowering a group of professionals to analyse AEACs and provide feedback to colleagues on the action plan.
These results are very encouraging. We were only able to identify 3 interventions in NHs that have been evaluated with respect to their impact on SC [27][28][29]. None of these studies highlighted significant changes in any dimension. In the case of healthcare facilities, several interventions were measured for their impact using a questionnaire similar to the NHSOPS [18,[29][30][31][32]. In general, earlier work only highlights significant changes in a few dimensions [30,31]. Scores for the dimensions Feedback and communication of errors and Overall perceptions of safety significantly improved in the studies by Ulibarrena et al. and Xie et al. [18,31], respectively. However, only Xie et al. [18] noted a significant improvement in 7 dimensions following a SC training program and a questionnaire the mainly targeted nurses.

Drivers to improve SC scores
Knowledge of the explanatory factors driving the change in scores is timely from the point of view of improving the system, and its large-scale deployment.
Quality policyemerged as a lever for the development of two dimensions (Overall perceptions of resident safety-organizational learning, and Staffing). It should be noted that the support system is based on tools, such as action plans and monitoring, that are usually used in Quality approaches. Furthermore, the fact that NHs are already familiar with this type of tool supports the successful implementation of the system. Being hospital-based contributed to improved scores for Dimension 4 (Supervisor expectations and actions promoting resident safety). In France, managers can be shared between NHs and hospitals. In this case, NHs naturally benefit from a more mature attitude to patient safety. Finally, the mean dependency score emerged as a factor in the evolution of SC, in particular Dimension 6 (Staffing). It should be noted that in France, an increase in the dependency of residents increases the financial aid paid to the NH, allowing the recruitment of more care professionals.
With respect to the implementation of the system, greater leadership among the steering group was consistent with improved perceptions of SC, particularly with regard to Dimension 1 (Overall perceptions of resident safety-organizational learning) and Dimension 5 (Compliance with procedures).
The literature highlights the main explanatory factors for a SC [19,28]. Two studies have examined the factors driving the evolution of scores: the first was conducted in Norwegian NHs [28], the second in a French healthcare institution [33]. Despite differences between the factors studied in these earlier works and the present study, past findings support our results. Our study shows that the successful implementation of the RM approach is conditional on the existence of a quality approach in the NH, and a steering committee that demonstrates leadership. These personnel have a mature understanding of SC [33], and appear to be the key to the success of the intervention, both in the short and the long term [34]. The organisational initiatives that they put in place, in collaboration with other professionals, are all-important [28]. The literature encourages top management to create and support such environments, which encourage staff to feel responsible for the SC of residents [28].

Strengths and limitations of the EHPAGE study
This research shows that this multimodal RM intervention, which ran over many months, was accepted and implemented by the majority of participants. The reasons for dropping out were independent of the scheme (notably due to issues linked to staff absences or changes in top management). The study also shows that the intervention can improve dimensions of SC among professionals working in NHs. Our results are all the more remarkable given that the study was conducted at the NH level (rather than the unit level, in a small number (58) of NHs, with a short period of time (18 months) between the two SC measures. Running the study over a period of at least three to five years [15], and with a larger sample of NHs could confirm our results. It would also allow us to improve the robustness of the explanatory factors, and avoid certain measurement hazards.
The questionnaire response rate (64% in 2016 and 56.5% in 2017), although comparable with other studies (66% [10], 69% [35]), may also constitute a selection bias. Due to financial and organisational constraints, researcher bias could not be avoided in this study. Finally, the fact that participation was voluntary, and that only two French départements were represented, means that the results cannot be extrapolated to the entire country.

The application of the EHPAGE project in the current health context
The EHPAGE intervention can be improved. Consideration could be given to the integration of residents' families, or their representatives, into the system. The failure to include these external actors in the process of improving healthcare safety and quality is all the more damaging as they have a different perspective, and often have an insight into more intimate aspects of residents' lives. In addition, health systems will evolve towards more home and outpatient care, requiring increased coordination between all actors. Therefore, and following Storm et al. [29], further thought needs to be given to a collaborative analysis that crosses organizational boundaries between different actors and structures, and examines how care can best be provided. Finally, the challenges associated with the quality of the working life of professionals who provide care should not be forgotten [36,37].

Conclusion
French NHs are in the early stages of implementing their RM approach. A regional structure, which was mandated to provide independent support in RM, designed a training package. This system, which is all the more important given the current COVID-19 pandemic, should ultimately improve the care of residents by developing a SC among NH professionals. Our findings show that the system was well-received, and implemented by participating NHs. Its impact was measured quantitively via the NHSOPS-F questionnaire. The analysis found that the improvement in Dimension 7 (Feedback and communication about incidents) was statistically significant, and that improvement in Dimension 1 (Overall perceptions of resident safety -organizational learning) was close to significant (+3.1%; p = 0.044). Our findings highlighted that the system was all the more effective in improving SC if the NH had already initiated a Quality approach. They also emphasize that it is important for top management to work with professionals who show leadership in the implementation of the RM approach.
To conclude, it is now up to each participating NH to keep the process alive. The system, which is now fully digitized, is currently being adapted to Belgian hospitals by a structure that is similar to QualiREL Santé.